feat(boson-avatar): add avatar plugin by warlockee · Pull Request #7019 · livekit/agents

warlockee · 2026-08-27T17:37:39Z

Summary

add livekit-plugins-boson-avatar, a BYOK Avatar plugin for Boson Higgs Audio-Driven Avatar
compose with the existing AgentSession voice pipeline instead of creating a second voice participant
accept provider connection settings from the host application; the plugin has no built-in provider hostname
add project-scoped Avatar discovery so applications can populate a face picker without hard-coded IDs
mint a least-privilege, room-scoped Avatar token and route generic 24 kHz agent PCM over LiveKit data streams
add provider start/delete lifecycle handling, idempotent retries, startup compensation, and cancellation-safe cleanup
register the package in the uv workspace and livekit-agents[boson-avatar] optional dependencies

Developer experience

The host application is the composition root: it supplies its Boson endpoint and API key, loads the project catalog server-side, and passes the Avatar selected by its UI to the session.

from livekit.plugins import boson_avatar

avatars = await boson_avatar.list_avatars()
# Return only avatar_id + name to the browser's face picker.

avatar = boson_avatar.AvatarSession(
    avatar_id=selected_avatar_id,
    api_url="https://your-boson-endpoint.example/v1/avatar/livekit",
)
await avatar.start(session, room=ctx.room)

api_url may alternatively be supplied through BOSON_AVATAR_API_URL; BOSON_API_KEY supplies the API key. The plugin intentionally does not define or assume a production hostname. It accepts the audio output of any compatible AgentSession voice pipeline and has no dependency on a Boson audio model. The URL must be an HTTPS base URL (loopback HTTP is allowed for local development); the plugin appends /avatars and the session resource paths. list_avatars() owns its short-lived HTTP client, so it also works in a normal application server outside a LiveKit job context.

Validation

uv run ruff format --check . and uv run ruff check . — passed
uv run python scripts/check_types.py — passed strict mypy
plugin unit suite — 27 tests plus 33 subtests passed, including catalog validation, endpoint validation, concurrent close, and repeated-cancellation regressions
wheel and sdist build passed
exact PR package completed a least-privilege live staging E2E: can_subscribe=False, Avatar joined, targeted PCM streamed, audio and video tracks published, and DELETE cleanup returned all four GPU workers to idle

Integration boundary

Deployment of a compatible Boson Avatar endpoint is an application/provider concern, not a plugin release dependency. The host app owns endpoint selection and composes the voice pipeline, Avatar plugin, LiveKit room, API credentials, and Avatar selection. The browser receives only catalog display data; the provider URL and Boson API key stay server-side.

The provider should retain its project-scoped catalog plus room-disconnect and max-duration reapers for process-kill and lost-response boundaries where a client can never receive the provider session ID.

CLAassistant · 2026-08-27T18:12:52Z

All committers have signed the CLA.

devin-ai-integration

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

warlockee · 2026-08-28T04:19:51Z

Ready for maintainer review at 30625f3.

All GitHub CI checks, CLA, and Devin Review are green.
Addressed and resolved all three security review threads. Provider errors are no longer emitted through exception tracebacks; logs retain only a safe error type, and provider identifiers are marked as PII.
Proactively stripped sensitive provider exception context after exhausted retries and covered compensation, shutdown, background-task, and retry logging with regression tests.
Local validation: 30 Boson Avatar tests passed, full repository make check passed across 650 typed source files, and both wheel and sdist built successfully.
The AvatarSession remains audio-provider agnostic and has no Boson audio dependency or hard-coded api.boson.ai endpoint.

@tinalenguyen, would you be able to take a look? You reviewed the recent Runway and Protoface Avatar plugin contributions.

warlockee · 2026-09-09T19:40:03Z

@tinalenguyen, following up on this Boson Avatar plugin contribution. I've merged current main (3334771d2) into the PR and resolved the dependency/workspace conflicts. The current head is 8e884f83d.

The latest code fixes both new public Devin findings: close can no longer spin on a completed startup-cleanup task while its callback is queued, and an avatar identity matching the agent is rejected before token issuance or provider creation. Regression tests reproduced both problems before the fixes and now pass, including DELETE retry after failed startup cleanup.

All GitHub checks on 8e884f83d are now passing (including Devin Review and CLA); three non-applicable jobs were skipped. All five public review threads are resolved.

Validation on the updated branch:

make check passed on Python 3.13 (663 typed source files).
Boson plugin suite: 32 tests and 38 subtests passed on Python 3.13 and 3.14.
Repository unit/audio end-of-turn suite after merging main: 2,758 passed, 5 skipped on Python 3.14.
Wheel and sdist built successfully; uv lock --check passed.

The three earlier public security review threads remain resolved. The plugin continues to compose with the existing AgentSession audio pipeline, with no Boson audio dependency or hard-coded provider endpoint.

Could you review this, or help route it to the appropriate avatar-plugin maintainer? I can address any further changes. If there are provider-onboarding or release requirements beyond the code review, please let me know what is needed. Devin also mentions a finding hidden by the repository's GitHub settings; if it remains actionable, could you share it here so I can address it?

warlockee · 2026-09-09T19:44:24Z

Hi @longcw, would you have time to review this Boson Avatar plugin when convenient? I noticed you reviewed the Spatius integration (#5821) and the recent Avatar audio-stream recovery fix (#7007), so your perspective on the session lifecycle and audio routing would be especially helpful.

The PR is updated with main and has no merge conflicts. CI, CLA, and Devin checks are passing at 8e884f83d; all five public review threads are resolved, with regression tests for the latest cleanup and identity-collision fixes. The validation details are in the comment above.

I'd appreciate any feedback and am happy to make changes to align with the existing Avatar plugins. If someone else is better placed to review this, a pointer would be very helpful. Thank you!

# Conflicts: # uv.lock

warlockee · 2026-09-18T00:26:06Z

Hi @livekit/agent-devs — could someone help route this PR to a specific maintainer for review, or let me know who should own it? The current head has no merge conflicts, and CI, CLA, and Devin Review are passing. I previously asked individual avatar-plugin maintainers, but I may be missing the right owner. If accepting a new third-party avatar plugin requires additional provider-onboarding steps or a reproducible live validation path, could you point me to those requirements? I can provide a staging walkthrough or test access privately and address any feedback. Thank you!

tinalenguyen · 2026-09-18T00:30:12Z

hi @warlockee, thanks for reaching out and being proactive! we have a queue for plugin submissions, could you submit this form so we can expedite the integration process?

warlockee · 2026-09-18T18:37:51Z

Thanks @tinalenguyen! We have submitted the LiveKit Plugin Partner Submission form for this Boson Avatar plugin. Please let us know if the team needs any additional details, a staging walkthrough, or test access during review. We are happy to help.

tinalenguyen

could we also bump the version to latest?

i also was not able to test this plugin, it seems that the realtime avatar service is still gated. i will follow up via email to facilitate access for the team

tinalenguyen · 2026-09-22T08:59:51Z

@@ -0,0 +1 @@
+


Suggested change

Thanks — removed the unnecessary py.typed marker in 23d4580. The Boson tests, Ruff, formatting, and package mypy checks pass.

Follow-up: I restored py.typed in a2179f8 because the repository type-check failed explicitly with “Package livekit.plugins.boson_avatar cannot be type checked due to missing py.typed marker.” The CI-style package-mode mypy check now passes for Boson. I’m reopening this thread so the required marker is clear.

devin-ai-integration

Devin Review found 1 new potential issue.

⚠️ 1 issue in files not directly in the diff

⚠️ Installed plugin loses type information

Without py.typed, type checkers treat the installed Boson plugin as untyped. Consumers lose checks for its annotated public API.

1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

warlockee · 2026-09-22T21:19:24Z

Thanks for flagging this. The py.typed marker is restored in a2179f8; removing it reproduced the package-discovery failure in scripts/check_types.py. The CI-style Boson package mypy check passes, and the fresh Python 3.10/3.13 type checks and all remaining CI checks are green.

warlockee · 2026-09-25T18:54:59Z

Hi @tinalenguyen — following up on the live test now that test access has been provided. Were you able to access the realtime Avatar service, or is anything still blocking the test? The requested code updates have been addressed and the latest CI is green. If you run into any access or setup issue, please let us know and we can help right away (and handle any credential details privately). Thank you!

warlockee added 2 commits August 27, 2026 17:36

feat(boson-avatar): add avatar plugin

7fc4721

fix(boson-avatar): harden session lifecycle

76b8f84

warlockee marked this pull request as ready for review August 27, 2026 18:18

warlockee requested a review from a team as a code owner August 27, 2026 18:18

devin-ai-integration Bot reviewed Aug 27, 2026

View reviewed changes

warlockee added 6 commits August 27, 2026 18:27

docs(boson-avatar): clarify provider-agnostic audio

d4b9431

Recover avatar sessions after LiveKit job restart

25f5ced

Derive UUID keys for avatar job recovery

9beebf4

Scope avatar recovery keys to session binding

e0b5c17

Complete avatar recovery binding

ca0e0ca

Decouple avatar plugin from provider endpoint

965852c

This comment was marked as resolved.

Sign in to view

Validate injected avatar API endpoints

619e063

This comment was marked as resolved.

Sign in to view

warlockee added 2 commits August 27, 2026 23:12

Add project-scoped avatar discovery

99f0ba8

Harden avatar provider error logging

30625f3

warlockee force-pushed the codex/boson-avatar-plugin branch from be915d5 to 30625f3 Compare August 28, 2026 04:13

Merge upstream main and resolve plugin registration conflicts

91724ee

This comment was marked as resolved.

Sign in to view

fix(boson-avatar): prevent cleanup spin and agent identity collision

8e884f8

warlockee added 2 commits September 11, 2026 23:25

Merge upstream main and resolve plugin dependency conflict

be97e1b

Merge remote-tracking branch 'origin/main' into codex/pr7019-monitor

0792bcf

# Conflicts: # uv.lock

This comment was marked as resolved.

Sign in to view

fix(boson-avatar): retain sessions after failed cleanup

bfdaae0

warlockee force-pushed the codex/boson-avatar-plugin branch from 99757f3 to bfdaae0 Compare September 15, 2026 16:59

Merge origin/main into codex/boson-avatar-plugin

568512c

Merge origin/main into codex/boson-avatar-plugin

0102778

tinalenguyen reviewed Sep 22, 2026

View reviewed changes

Address Boson avatar review feedback

23d4580

devin-ai-integration Bot reviewed Sep 22, 2026

View reviewed changes

Restore Boson typing marker for CI

a2179f8

Merge origin/main into codex/boson-avatar-plugin

d7a1a2d

Conversation

warlockee commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Developer experience

Validation

Integration boundary

Uh oh!

CLAassistant commented Aug 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

devin-ai-integration Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

🔍 Devin Review: 1 flag

Uh oh!

This comment was marked as resolved.

Uh oh!

This comment was marked as resolved.

Uh oh!

warlockee commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as resolved.

Uh oh!

warlockee commented Sep 9, 2026

Uh oh!

warlockee commented Sep 9, 2026

Uh oh!

This comment was marked as resolved.

Uh oh!

warlockee commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tinalenguyen commented Sep 18, 2026

Uh oh!

warlockee commented Sep 18, 2026

Uh oh!

tinalenguyen left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tinalenguyen Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

warlockee Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

warlockee Sep 22, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

⚠️ Installed plugin loses type information

Uh oh!

warlockee commented Sep 22, 2026

Uh oh!

warlockee commented Sep 25, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

warlockee commented Aug 27, 2026 •

edited

Loading

CLAassistant commented Aug 27, 2026 •

edited

Loading

devin-ai-integration Bot left a comment •

edited

Loading

warlockee commented Aug 28, 2026 •

edited

Loading

warlockee commented Sep 18, 2026 •

edited

Loading